Papers with attention-based model
Zero Pronoun Resolution with Attention-based Neural Network (C18-1)
Copied to clipboard
| Challenge: | Recent neural network methods for zero pronoun resolution use contextual information to encode the zero pronomins since they contain no actual content. |
| Approach: | They propose a self-attention mechanism for encoding zero pronouns that focus on some informative parts of the associated texts and produce an efficient way of encode them. |
| Outcome: | The proposed model significantly surpasses existing Chinese zero pronoun resolution baseline systems. |
AMR Parsing as Sequence-to-Graph Transduction (P19-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) parsing is the task of transducing natural language text into AMR, a graphbased formalism used for capturing sentence-level semantics. |
| Approach: | They propose a model that treats AMR parsing as sequence-to-graph transduction by aligner-free, and can be effectively trained with limited amounts of labeled AMR data. |
| Outcome: | The proposed model outperforms all previously reported SMATCH scores on AMR 2.0 (76.3%) and AMR 1.0 (70.2%). |
ZEST: Zero-shot Learning from Text Descriptions using Textual Similarity and Visual Summarization (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Specifically, given birds’ images with free-text descriptions of their species, we learn to classify images of previously-unseen species based on specie descriptions. |
| Approach: | They propose to leverage the similarity between species and extract visual summaries from the texts to match visual features to the parts of the text that discuss them. |
| Outcome: | The proposed model outperforms the state-of-the-art on the largest benchmarks for text-based zero-shot learning. |
Effective Use of Context in Noisy Entity Linking (D18-1)
Copied to clipboard
| Challenge: | Effectively using an entity mention's context to disambiguate it is the crux of the entity linking task. |
| Approach: | They propose to use convolutional neural networks to extract cues from context to effectively disambiguate between closely related concepts. |
| Outcome: | The proposed model outperforms previous work on the WikilinksNED test set by 2.8% absolute. |
Situated Mapping of Sequential Instructions to Actions with Single-step Reward Observation (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to map context-dependent sequential instructions to actions are based on discourse and state dependencies . we evaluate on SCONE domains and show absolute accuracy improvements of 9.8%-25.3% . |
| Approach: | They propose a model that considers previous utterances and the state of the world to map sequential instructions to actions. |
| Outcome: | The proposed model improves on the SCONE domains and on the target domains. |
Commonsense Justification for Action Explanation (D18-1)
Copied to clipboard
| Challenge: | a recent study examines the commonsense reasoning used by humans to justify an AI prediction. |
| Approach: | They propose an approach that models object relations/attributes of the world as latent variables and jointly learns a performer that predicts actions and an explainer that gathers commonsense evidence to justify the action. |
| Outcome: | The proposed model achieves significantly higher performance in both action prediction and justification. |
Data-Informed Global Sparseness in Attention Mechanisms for Deep Neural Networks (2024.lrec-main)
Copied to clipboard
| Challenge: | Attention pruning techniques have been developed to identify and exploit sparseness . previous work has taken pioneering steps to discover and explain the sparsity in attention patterns . |
| Approach: | They propose a framework that observes attention patterns in a fixed dataset and generates a global sparseness mask. |
| Outcome: | The proposed approach saves 90% of computations and maintains quality of results. |
Learning Attention-based Embeddings for Relation Prediction in Knowledge Graphs (P19-1)
Copied to clipboard
| Challenge: | Existing knowledge graphs (KGs) are incomplete or partial information, in the form of missing relations between entities, which gives rise to the task of knowledge base completion (also known as relation prediction). |
| Approach: | They propose to capture both entity and relation features in any given neighborhood and encapsulate relation clusters and multi-hop relations in their attention-based model. |
| Outcome: | The proposed model captures both entity and relation features in any given neighborhood and also encapsulates relation clusters and multi-hop relations. |
Long Sequence Modeling with Attention Tensorization: From Sequence to Tensor Learning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a lack of attention-based models for long sequences poses challenges for long-sequence modeling . attention tensorization can be used to extend context lengths with improved efficiency . tenorization enables training of LLMs with context length longer than those trained on . |
| Approach: | They propose to tensorize long input sequences into compact tenses followed by attention on each transformed dimension. |
| Outcome: | The proposed model can be used as efficient transformer backbones to extend input context length with improved memory and time efficiency. |